Comparative effectiveness of neutralising monoclonal antibodies in high risk COVID-19 patients: a Bayesian network meta-analysis

The purpose of this work was to review and synthesise the evidence on the comparative effectiveness of neutralising monoclonal antibody (nMAB) therapies in individuals exposed to or infected with SARS-CoV-2 and at high risk of developing severe COVID-19. Outcomes of interest were mortality, healthcare utilisation, and safety. A rapid systematic review was undertaken to identify and synthesise relevant RCT evidence using a Bayesian Network Meta-Analysis. Relative treatment effects for individual nMABs (compared with placebo and one another) were estimated. Pooled effects for the nMAB class compared with placebo were estimated. Relative effects were combined with baseline natural history models to predict the expected risk reductions per 1000 patients treated. Eight articles investigating four nMABs (bamlanivimab, bamlanivimab/etesevimab, casirivimab/imdevimab, sotrovimab) were identified. All four therapies were associated with a statistically significant reduction in hospitalisation (70–80% reduction in relative risk; absolute reduction of 35–40 hospitalisations per 1000 patients). For mortality, ICU admission, and invasive ventilation, the risk was lower for all nMABs compared with placebo with moderate to high uncertainty due to small event numbers. Rates of serious AEs and infusion reactions were comparable between nMABs and placebo. Pairwise comparisons between nMABs were typically uncertain, with broadly comparable efficacy. In conclusion, nMABs are effective at reducing hospitalisation among infected individuals at high-risk of severe COVID-19, and are likely to reduce mortality, ICU admission, and invasive ventilation rates; the effect on these latter outcomes is more uncertain. Widespread vaccination and the emergence of nMAB-resistant variants make the generalisability of these results to current patient populations difficult.


Overview
Where feasible, formal evidence synthesis in the form of network meta-analysis (NMA) will be carried out. All outcomes extracted as part of the rapid review will be synthesised where possible.

Identification of Evidence Networks
Since the included studies are expected to report different outcomes, separate outcomespecific evidence networks will be constructed. Studies enrolling outpatients infected with or exposed to Covid-19 will be considered, while studies enrolling patients already hospitalised as a result of Covid-19 will not be included for the following reasons: • Patient characteristics are likely to differ considerably, thus there is a high risk of effect modification if trials involving hospitalised patients are synthesised with those enrolling only outpatients • The timing of the interventions relative to symptom onset is likely to differ considerably (hospitalised patients will receive treatment much later on average) Different doses of the same mAB therapy will be treated as a single intervention in the base case. This decision was made in advance due to the expectation that the number of included studies would be small, and that certain event types (e.d. deaths, ICU admissions) were expected to be rare. This approach increases statistical power at the potential cost of additional heterogeneity (the latter will be accounted for via the use of random-effects models where possible).
Subgroup analysis may be carried out for the following groups if data is available: The feasibility of including each study in the relevant evidence network will be then be assessed according to the following criteria: • Comparability of baseline patient characteristics and inclusion criteria across trials in the network • Consistency in the definitions and reporting of outcomes • For dichotomous outcomes: at least one event reported across treatment arms

Extraction of outcome data
All available outcome data for studies meeting the PICO criteria will be extracted (including subgroup data). Separate outcome data for each treatment arm will be extracted where available, which is expected to consist of: • Dichotomous outcomes: number of events and sample size by treatment arm • Continuous outcomes: mean and standard error by treatment arm. Where standard error is not reported, this will be estimated from published confidence intervals or similar.
It is not anticipated that any other types of data (e.g. time to event, count data) will arise. For publications reporting only summary measures of treatment effect, these will also be extracted, together with the associated measures of uncertainty (e.g. standard errors and/or confidence intervals) for comparison purposes.

Statistical analysis
The following treatment effects will be synthesised: • Relative risks for dichotomous outcomes • Mean differences and/or mean ratios for continuous outcomes (the preferred measure of treatment effect will be context-specific) Bayesian network meta-analysis will be used to synthesise treatment effects (Dias et al., 2013a). For dichotomous outcomes, a GLM with a binomial likelihood and a log link function will be used (Warn et al., 2002), while for continuous outcomes a normal likelihood and an identity link will be used. It anticipated that for the base case, random-effects meta-analysis will be used, except where there is only a single trial contributing to each comparison and no closed loops.
It is anticipated that the NMAs will include a small number of studies and that event counts will be low within each study. Therefore, for dichotomous outcomes, a normal distribution with mean 0 and standard deviation 2.82 will be used as a prior distribution for the treatment effect parameters. This is the weakly informative prior distribution recommended in (Günhan et al., 2020) for meta analysis of few studies of rare events, which is symmetric about the null (i.e. neither benefit nor harm is regarded as more likely a priori) and has 95% of the density between 1/250 and 250 (i.e. a 95% prior belief that no relative risk exceeds 250).
Due to the small number of studies in the network, weakly informative priors will be used for the heterogeneity parameters (HN(0.5) for relative risks) as discussed in (Röver et al., 2021) and elsewhere. Sensitivity analysis will be carried out on prior distributions.
Analysis will be carried out in R, primarily using the package BUGSnet (Béliveau et al., 2019) (which implements MCMC sampling in JAGS). MCMC sampling wil be conducted using 100,000 iterations of which 50,000 are burn-in iterations, and 5,000 adaptation iterations, and 3 chains.
Convergence and model plausibility will be assessed by checking: • Heterogeneity will primarily be assessed qualitatively via comparisons of the characteristics of the included studies (patients, outcomes etc.). Where differences in patient characteristics exist across studies, the potential for effect-modification will be assessed by examining subgroup analysis if included in the identified studies.
Consistency will be assessed by comparing direct and indirect estimates of treatment effects.
Estimated treatment effects will also be compared with estimates of the between-trial standard deviation parameter tau (where random-effects models have been used). As the evidence networks are not anticipated to contain a large number of comparisons, formal statistical methods to assess inconsistency and heterogeneity will not be carried out.

Calculation of absolute effects
In the case of dichotomous outcomes, baseline natural history models will also be fitted to the event rates in the SoC arms of the included studies. Again a GLM with a binomial likelihood and log link function will be used, with a preference for random-effects models where possible. These will be used to estimate the absolute event rates and risk reductions versus SoC for each treatment and each outcome.

Sensitivity analysis
The following sensitivity analysis will be carried out: 4. Where random effects models have been used in the base case, the following alternative priors will be used for the heterogeneity parameter: informative priors from (Turner et al., 2012) [log-normal(-4.06,1.45^2) for mortality and log-normal(-3.02, 1.85^2) for other outcomes] -Non-informative prior U(0,5) -Alternative weakly informative prior Half-Cauchy(0.5) (Röver et al., 2021) Fixed-effects models will also be fitted.

Pooled mAB pairwise meta-analysis
We estimated the pooled mAB versus placebo effect by reclassifying all mAB therapies as a single intervention and carrying out the corresponding pairwise meta-analysis as before.
For all outcomes we investigate the following alternative priors for the treatment effect parameters: 1. More informative prior: N(0,1.175^2). This distribution gives a 95% prior probability that no relative risk between any pair of treatments in the network exceeds 10 (note that 0.175=log(10)/1.96).
2. Less informative prior: N(0,4.700^2). This distribution gives a 95% prior probability that no relative risk between any pair of treatments in the network exceeds 10,000 (note that 4.700=log(10000)/1.96).
# This code runs sensitivity analysis on the prior distributions for the rela tive treatment effects and saves the results.

Comparison of fixed and random effects.
# Sensitivity analysis to compare fixed versus random effects. # Run alternative NMA model (typically fixed effects, where base case is rand om effects) and save NMA output # Note that model is only run if output file does not exist already (delete t o re-run) for (i in seq_along(outcomes)) { # Character string for alternative model type alt_type<-ifelse(outcomes_full$type[i]=="re","fe","re") # File path to save NMA output alt_file<-paste0("./output_data/",outcomes[i],"_NMA_",alt_type,".RDS") } } # Vector of captions for the output figures captions_fere<-paste0(c("Comparison of the fit of fixed and random effects mo dels for the outcome of ", "Comparison of estimated treatment effects (relative ri sks) versus placebo for the outcome of "), rep(outcomes_full$Outcome,times=1,each=2))

Heterogeneity parameter prior sensitivity analysis
For all outcomes we examined the sensitivity of the results to the choice of prior distribution for the heterogeneity parameter (between trial standard deviation). The following distributions were examined: 1. Base case: half-normal(0.5) 2. Uniform(0,5) 3. Informative priors from Turner et al. (2012) for the between trial variance: log-normal(-4.06,1.45^2) for mortality and log-normal(-3.02, 1.85^2) for other outcomes 4. Alternative weakly-informative prior: Half-Cauchy(0.5) Röver et al. (2021) Turner et al. (2012) gives prior for sigma^2, therefore it is necessary to edit the JAGS code produced by BUGSnet to implement these in a scenario analysis. This is done in the following code chunk.

Investigation of consistency
The only closed loops in the evidence networks arise from the direct comparison between bamlanivimab/etesevimab and casirivimab/imdevimab carried out in (McCreary et al., 2021).
The outcomes affected are mortality, hospitalisation, infusion-related AEs and serious AEs. Consistency was assessed by comparing the direct estimate of relative effects of ballanivimab/etesevimab versus casirivimab/imdevimab from (McCreary et al., 2021), and the indirect estimate of the same effect obtained re-running an NMA of all studies evaluating bamlanivimab/etesevimab and casirivimab/imdevimab with (McCreary et al., 2021) excluded (note that the latter does not contribute to the indirect estimate as there is no placebo arm).
4. Hospital Length-of-Stay was reported in three publications. Due to time-constraints and model complexity (i.e. the need to account for zero-inflated data) an NMA for this outcome has not been carried out.
5. For the supplemental oxygen outcome, only one study (Gupta et al., 2021) reported results in the base case ( (Eom et al., 2021) also reported results but this study did not meet the inclusion criteria of the Review). This outcome has not been included in the analysis.

Inclusion of regdanvimab as intervention
In this section we explore the effect of including results from (Eom et al., 2021), which compared regdanvimab and placebo in a mixed-risk population, in the NMA. This study was excluded during the rapid review as results were not reported separately for the high-risk subpopulation. It has been included as a scenario analysis as it has been licensed in the EU for the population of interest to this review (adults with COVID-19 who are at increased risk of progressing to severe COVID-19). The study contained three arms: regdanvimab 40mg/kg, regdanvimab 80mg/kg, and placebo. For consistency with other comparisons in the network, outcomes for regdanvimab were pooled across 40mg/kg and 80mg/kg arms.
All relevant outcomes were considered for inclusion in this scenario analysis: • Mortality: excluded as no events in either arm • Hospitalisation: included • Invasive ventilation: excluded as no events observed in placebo arm (1 event in treatment arm), therefore it was not possible to estimate a relative-risk for regdanvimab versus placebo for this outcome. While an RR for placebo versus regdanvimab could in theory be estimated this would be highly uncertain due to there being only a sinlge event in the regdanvimab arm.
• ICU admission: excluded as no events in either arm

Absolute Risk Reductions
For all dichotomous outcomes, baseline event rates under SoC have been estimated by pooling the rates from the SoC arms across trials using a Bayesian random-effects model, analogous to the model used for relative treatment effects but with a random intercept term only (see NICE TSD 5 (Dias et al., 2013b)). In all cases, event rates from (Eom et al., 2021) have been excluded from this calculation as this study included non-high-risk patients, who are likely to exhibit lower baseline event rates for most outcomes compared with the target population.
Absolute risks and risk differences for regdanvimab have therefore been estimated by applying the relative effects for regdanvimab versus placebo, to baseline event rates estimated from the remaining studies. This approaches assumes that the relative effect of regdanvimab versus placebo will generalise to the higher-risk populations of the other studies.

Treatment effects prior
For all outcomes we investigate the following alternative priors for the treatment effect parameters: 1. More informative prior: N(0,1.175^2). This distribution gives a 95% prior probability that no relative risk between any pair of treatments in the network exceeds 10 (note that 0.175=log(10)/1.96).
2. Less informative prior: N(0,4.700^2). This distribution gives a 95% prior probability that no relative risk between any pair of treatments in the network exceeds 10,000 (note that 4.700=log(10000)/1.96).

Heterogeneity Prior
For all outcomes we examined the sensitivity of the results to the choice of prior distribution for the heterogeneity parameter (between trial standard deviation). The following distributions were examined: 1. Base case: half-normal(0.5) 2. Uniform(0,5) 3. Informative priors from Turner et al. (2012) for the between trial variance: log-normal(-4.06,1.45^2) for mortality and log-normal(-3.02, 1.85^2) for other outcomes 4. Alternative weakly-informative prior: Half-Cauchy(0.5) Röver et al. (2021) Figure 4.49: Estimated treatment effects for each mAB versus placebo obtained from varying the prior distribution on the heterogeneity parameter sigma, for the outcome of Hospitalisation              Figure 4.52: Estimated treatment effects for each mAB versus placebo obtained from varying the prior distribution on the heterogeneity parameter sigma, for the outcome of Infusion-related AEs    For all three outcomes, 95% credible intervals for the direct and indirect relative treatment effect estimates overlap, inidicating that there is no statistically significant inconsistency in the networks. The results show a numerical difference between the direct and indirect effect estimates for the mortality outcome, which could potentially indicate a violation of the assumption of consistency which was not detected due to low statistical power. Upon examining the raw event counts, there were no deaths in the bamlanivimab/etesevimab arms of Dougan et al. (2021a), compared with one death in each casirivimab/imdevimab arm of (Weinreich et al., 2021). When combined with the weakly informative prior, this results in a very high point estimate of the RR of mortality for casirivimab/imdevimab versus bamlanivimab/etesevimab, although this estimate is highly uncertain due to extremely small event numbers. By contrast, mortality rates observed in (McCreary et al., 2021) were similar for both treatments, giving a relative risk that is close to 1. Due to the low event numbers it is not possible to determine whether this discrepancy has arisen by chance, or indicates a genuine violation of the consistency assumption. This situation is similar for the SAE outcome.

Serious AEs
The results for hospitalisation do not indicate any violation of the consistency assumption.